REMARKS 

Applicant respectfully requests reconsideration and allowance of the 
subject application. Claim 87 is canceled without prejudice. Claim 82 is amended 
to incorporate the elements of canceled claim 87. Claim 88 is amended to depend 
from claim 82 rather than canceled claim 87. Claims 1-6, 12, 13, 28-86, and 88-91 
are pending in this application. 

Double Patenting 

Claims 1-6, 12-13, and 28-91 stand rejected under the judicially created 
doctrine of obviousness-type double patenting as being unpatentable over claims 
1-18 of U.S. Patent No. 6,006,241 to Purnaveja et aL, claims 1-18 of U.S. Patent 
No. 6,230,172 to Purnaveja et aL, claims 1-39 of U.S. Patent No. 6,173,317 to 
Chaddha et aL, and claims 1-25 of U.S. Patent No. 6,484,156 to Gupta et aL 
Applicant previously submitted a terminal disclaimer to overcome the 
obviousness-type double patenting rejection over the claims of U.S. Patent No. 
6,006,241 to Purnaveja et al. Accompanying this response are terminal 
disclaimers in compliance with 37 CFR 1.321(c) to overcome the obviousness- 
type double patenting rejections over claims 1-18 of U.S. Patent No. 6,230,172 to 
Purnaveja et aL, claims 1-39 of U.S. Patent No. 6,173,317 to Chaddha et aL, and 
claims 1-25 of U.S. Patent No. 6,484,156 to Gupta et al. 

35 U.S.C. § 102 

Claims 1-6, 12-13, and 28-91 stand rejected under 35 U.S.C. § 102(e) as 
being unpatentable over "Towards Intelligent Recognition of Multimedia Episodes 
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in Real-Time Applications" 1994 ACM 0-89791-686-7/94/0010 by Gabbe et al. 
(hereinafter "Gabbe"). Claim 87 has been canceled without prejudice. Applicant 
respectfully submits that claims 1-6, 12, 13, 28-86, and 88-91 are not anticipated 
by Gabbe. 

Gabbe is directed to automatically generating indexes of real-time streams 
without requiring deep content analysis (see, Abstract). For each type of recorded 
system element (applications, users, video streams, audio streams, and 
communications & control), a set of machine identifiable events or conditions 
called triggers is specified (see, p. 228, middle of col 1; and Figure 1). These 
triggers are events or conditions whose occurrence causes the recording system to 
create a record describing some aspects of the current state of the corresponding 
element (see, p. 228, middle of col. 1). These records contain an IER (Iconic 
Episode Representative) associated with the event (see, Figure 1). The IERs are 
mouse- sensitive icons that are symbolic or visual (see, p. 229, bottom of col. 1 and 
top of col. 2). 

By putting sequences of related IERs into a structure - such as a linear 
array - one can provide a rudimentary "visual map" of the history of an interaction 
(see, p. 229, middle of col. 1). Such a Table of IERs, or TIER, gives a visually 
oriented outline of a complex series of episodes (see, p. 229, middle of col. 1), 
where episodes refer to a "unit of meaning" that exists as a distinguishable part of 
a large, multi- faceted, temporally extended entity, such as a meeting, a lecture, or 
an interactive game (see, p. 227, top of col. 2). 

The TIERs are used in conjunction with a query-based information retrieval 
interface to provide multimodal retrieval capabilities (see, p. 232, bottom of col. 
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1). A recording agent builds a TIER for a conference and an index of all the text, 
control, and event information that can be obtained (see, p. 232, top of col. 2). 
Generating the TIER is discussed on page 233, top of col. 1, as follows: 

To generate a TIER, the recording agent has a number of 
processes that monitor each media stream and automatically 
partition them into self-contained units based on the contents. The 
recording agent then generates a icon of 160x120 pixels 
summarizing each visually-oriented episode. For video, the icon is a 
snapshot taken from the stream itself. The agent builds the TIER in 
real-time, and makes it immediately available to participants for 
browsing through the meeting and replaying earlier sections of the 
meeting. This is particularly useful for late joiners. By default, the 
visible part of a TIER always contains the most recently generated 
icons. However, the user may scroll back and forth in it and peruse 
any segment by clicking on an IER. In addition, the user may select 
any IER and annotate it with text using the annotation facility (see 
Figure 5). 

In contrast, claim 1 is directed to one or more computer-readable media 
containing a computer program for annotating streaming media, wherein the 
program performs steps comprising: 

creating annotations interactively with a user, wherein the 
annotations correspond to identified segments of one or more media 
streams; 

graphically ordering the annotations in a desired order of 
presentation in response to user input; and 

in response to a user instruction, sequentially presenting the 
annotations along with their corresponding identified media stream 
segments in the desired order of presentation. 

Applicant respectfully submits that Gabbe does not disclose graphically ordering 
the annotations in a desired order of presentation in response to user input as 
recited in claim 1. 
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As discussed above, Gabbe mentions that a user may select an IER and 



annotate it with text using an annotation facility. However, as discussed above, 
these IERs are associated with events or conditions whose occurrence causes the 
recording system to create a record describing some aspects of the current state of 
the corresponding element. The order of the IERs is driven by these events or 
conditions, and it is these IERs that Gabbe mentions may be annotated by a user. 
The ordering of any annotations that may be generated in response to the user 
annotating an IER will be based on the ordering of the IERs, not on any desired 
order of presentation. 

In other words, there is nothing in Gabbe that discloses that annotations can 
be ordered in any desired order. There is nothing in Gabbe that discloses that 
annotations can be placed in any order other than the order of the IERs. 

In the March 23 Final Office Action at page 3, the following portion of 
Gabbe is cited (spanning p. 229, bottom of col. 2 to p. 230, top of col. 1): 

As discussed in the subsequent section on multimedia-conference- 
recording application, the icons may be presented to the user as a 
film strip (iconic slider) through which the user can scroll or jump 
(see Figure 2). Since each IER is associated with a 
Last_Record_Generated vector, a user can redisplay all or part of the 
entire state of the recorded system associated with the icon by 
"selecting" it. The icons also serve as easily identifiable anchor 
points for attaching annotations, control events, and index records to 
episodes. 

It was also asserted in the March 23 Final Office Action at pages 3-4 that: 

The IERs are displayed in some kind of window, allowed by 
window system (Figure 2) (e.g. Microsoft windows programming, 
Motif GUI standards in UNIX or Java GUI standard). Dragging is 
allowed by all of the above GUI standards. Therefore a user can 
drag the icons to reorient them along the desired sequence and replay 
the interesting portion/clips of the conference in the desired order. 
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User also can select particular icons by using a mouse. Dragging 
visual entities in a window is a very basic feature of a GUI window. 

Thus, it appears that the March 23 Final Office Action is asserting that 
because dragging is allowed in various GUI standards, the user can drag the icons 
in Gabbe to reorient them along the desired sequence and replay the interesting 
portion/clips of the conference in the desired order. Applicant respectfully 
submits that nothing in Gabbe discloses any such notion. Nowhere in Gabbe is 
there any mention or discussion of a user being able to reorient or drag such icons. 
Without such mention or discussion, Applicant respectfully submits that Gabbe 
cannot disclose graphically ordering the annotations in a desired order of 
presentation in response to user input as recited in claim 1. 

MPEP §2131 states that, "A claim is anticipated only if each and every 
element as set forth in the claim is found, either expressly or inherently described, 
in a single prior art reference," Verdegaal Bros. v. Union Oil Co. of California, 
814 F.2d 628, 631, 2 USPQ2d 1051, 1053 (Fed. Cir. 1987). Applicant respectfully 
submits that the rejection of claim 1 does not meet this standard. As there is not 
even a mention in Gabbe of graphically ordering the annotations in a desired order 
of presentation in response to user input, much less a description of graphically 
ordering the annotations in a desired order of presentation in response to user 
input, Applicant respectfully submits that the standard for anticipation under 35 
U.S.C. §102 is not satisfied by Gabbe with respect to claim 1. 

Thus, for at least these reasons, Applicant respectfully submits that claim 1 
is allowable over Gabbe. 
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With respect to claim 12, Applicant respectfully submits that Gabbe does 
not disclose allowing the ordering of the plurality of annotation identifiers to be 
changed by the user as recited in claim 12. 

Applicant respectfully submits that, similar to the discussion above 
regarding claim 1, there is no ordering of annotation identifiers that can be 
changed by the user in Gabbe. As discussed above, the TIER of Gabbe gives a 
visually oriented outline of a complex series of episodes, but nowhere does Gabbe 
have any discussion that the user can change the ordering of episodes in the TIER, 
much less change the ordering of annotation identifiers. Furthermore, nowhere in 
Gabbe is there any mention that one or more of a plurality of annotations and a 
portion of the media stream corresponding to each of the plurality of annotations is 
provided in an order defined by the user-changeable ordering of annotation 
identifiers as recited in claim 12. 

Thus, for at least these reasons, Applicant respectfully submits that claim 
12 is allowable over Gabbe. 

Regarding claim 28, Applicant respectfully submits that, similar to the 
discussion above regarding claim 12, Gabbe does not disclose or suggest 
reordering the plurality of identifiers in accordance with user input to change the 
order in which the media segments are to be presented as recited in claim 28. 

Thus, for at least these reasons, Applicant respectfully submits that claim 
28 is allowable over Gabbe. 

Regarding claim 34, Applicant respectfully submits that, similar to the 
discussion above regarding claim 1, Gabbe does not disclose or suggest 
graphically ordering annotations in a desired order of presentation in response to 
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user input, wherein the annotations correspond to identified segments of one or 
more media streams; and in response to a user instruction, sequentially presenting 
the annotations along with their corresponding identified media stream segments 
in the desired order of presentation as recited in claim 34. Thus, for at least these 
reasons, Applicant respectfully submits that claim 34 is allowable over Gabbe. 

Regarding claim 38, Applicant respectfully submits that, similar to the 
discussion above regarding claim 12, Gabbe does not disclose or suggest 
configuring a first portion of a user interface to display a plurality of identifiers 
corresponding to a plurality of annotations, the plurality of identifiers 
corresponding to a playlist of media segments to be seamlessly presented to a user; 
and reordering the plurality of identifiers in accordance with user input to change 
the order in which the media segments are to be presented as recited in claim 38. 
Thus, for at least these reasons, Applicant respectfully submits that claim 38 is 
allowable over Gabbe. 

Regarding claim 42, Applicant respectfully submits that, similar to the 
discussion above regarding claim 1, Gabbe does not disclose or suggest 
graphically ordering annotations in a desired order of presentation in response to 
user input, wherein the annotations correspond to identified segments of one or 
more media streams; and in response to a user instruction, sequentially presenting 
the annotations in the desired order of presentation as recited in claim 42. Thus, 
for at least these reasons, Applicant respectfully submits that claim 42 is allowable 
over Gabbe. 

Regarding claim 43, Applicant respectfully submits that, similar to the 
discussion above regarding claim 1, Gabbe does not disclose or suggest 
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graphically ordering annotations in a desired order of presentation in response to 
user input, wherein the annotations correspond to identified segments of one or 
more media streams; and in response to a user input, sequentially presenting the 
identified media segments corresponding to the annotations in the desired order of 
presentation as recited in claim 43. Thus, for at least these reasons, Applicant 
respectfully submits that claim 43 is allowable over Gabbe. 

Regarding claim 44, Applicant respectfully submits that, similar to the 
discussion above regarding claim 12, Gabbe does not disclose or suggest means 
for configuring a first portion of a user interface to display a plurality of identifiers 
corresponding to a plurality of annotations, the plurality of identifiers 
corresponding to a playlist of media segments to be seamlessly presented to a user; 
and means for reordering the plurality of identifiers in accordance with user input 
to change the order in which the media segments are to be presented as recited in 
claim 44. Thus, for at least these reasons, Applicant respectfully submits that 
claim 44 is allowable over Gabbe. 

Regarding claim 48, Applicant respectfully submits that, similar to the 
discussion above regarding claim 1, Gabbe does not disclose or suggest 
graphically ordering the annotations in a desired order of presentation in response 
to user input; and in response to a user instruction, sequentially presenting the 
annotations along with their corresponding identified media stream segments in 
the desired order of presentation as recited in claim 48. Thus, for at least these 
reasons, Applicant respectfully submits that claim 48 is allowable over Gabbe. 

Regarding claim 52, Applicant respectfully submits that, similar to the 
discussion above regarding claim 1 , Gabbe does not disclose or suggest a plurality 
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of instructions that cause a system to graphically order the annotations in a desired 
order of presentation in response to user input; and in response to a user 
instruction, sequentially present the annotations along with their corresponding 
identified media stream segments in the desired order of presentation as recited in 
claim 52. Thus, for at least these reasons, Applicant respectfully submits that 
claim 52 is allowable over Gabbe. 

Regarding claim 54, Applicant respectfully submits that, similar to the 
discussion above regarding claim 1, Gabbe does not disclose or suggest receiving 
an indication of a plurality of annotations selected by a user, wherein each of the 
plurality of annotations corresponds to a media stream or to one or more media 
streams; and seamlessly providing, in an order which is identified by the user and 
can be changed by the user, one or more of, the plurality of annotations, and at 
least a portion of the media stream corresponding to each of the plurality of " 
annotations as recited in claim 54. Thus, for at least these reasons, Applicant 
respectfully submits that claim 54 is allowable over Gabbe. 

Regarding claim 59, claim 59 depends from claim 54 and Applicant 
respectfully submits that claim 59 is allowable over Gabbe for at least the reasons 
discussed above with respect to claim 54. Furthermore, Applicant respectfully 
submits that Gabbe does not disclose or suggest storing the at least a portion of the 
media stream corresponding to each of the plurality of annotations as a new media 
stream of the one or more media streams as recited in claim 59. There is no 
discussion whatsoever in Gabbe of storing portions of a media stream 
corresponding to each of a plurality of annotations as a new media stream . 
Nothing in Gabbe makes any disclosure or suggestion of storing such a new media 
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stream that is based on the plurality of annotations as recited in claim 59. Thus, 
for at least these reasons, Applicant respectfully submits that claim 59 is allowable 
over Gabbe. 

Regarding claim 63, Applicant respectfully submits that, similar to the 
discussion above regarding claim 1, Gabbe does not disclose or suggest retrieving 
a particular collection of annotations from the annotation database; presenting the 
annotations of the retrieved collection to a user in an order which is input by the 
user and which can be changed by the user; and managing sequential presentation 
to the user of the media stream segments corresponding to the presented 
annotations as recited in claim 63. Thus, for at least these reasons, Applicant 
respectfully submits that claim 63 is allowable over Gabbe. 

Regarding claim 68, claim 68 depends from claim 63 and Applicant 
respectfully submits that claim 68 is allowable over Gabbe for at least the reasons 
discussed above with respect to claim 63. Furthermore, Applicant respectfully 
submits that, similar to claim 59 discussed above, Gabbe does not disclose or 
suggest saving information regarding the media stream segments as an additional 
new media stream as recited in claim 68. Thus, for at least these reasons, 
Applicant respectfully submits that claim 68 is allowable over Gabbe. 

Regarding claim 73, claim 73 recites, in part: 

receiving an indication of particular user-selected annotations 
of a plurality of annotations, wherein each of the plurality of 
annotations corresponds to a media stream or to one or more media 
streams; and 

seamlessly providing one or more of, 

the user-selected annotations, and 
at least a portion of the media stream corresponding to 
each of the user-selected annotations. 
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Applicant respectfully submits that Gabbe does not disclose or suggest receiving 
an indication of particular user-selected annotations of a plurality of annotations, 
wherein each of the plurality of annotations corresponds to a media stream or to 
one or more media streams, and seamlessly providing one or more of the user- 
selected annotations and at least a portion of the media stream corresponding to 
each of the user selected annotations as recited in claim 73. 

In the March 23 Final Office Action at page 3, it was asserted that: 

The quoted text excerpts clearly indicates that multimedia streams 
are divided into episodes. The episode consists of a plurality of 
continuous/seamless frames. The IER is associated with an episode. 
The annotations can be typed in an IER. Therefore IER is basically 
a visual annotation to the associated episode . . . Within each episode 
the playback is seamless. 

However, Applicant respectfully submits that nothing in Gabbe discloses or 
suggests the ability of a user to select particular annotations from a plurality of 
annotations corresponding to a media stream, and then seamlessly providing those 
user-selected annotations and/or the portions of the media stream corresponding to 
those user-selected annotations. There is no mention or discussion whatsoever in 
Gabbe of seamlessly providing such annotations and/or portions of the media 
stream corresponding to such annotations. Even if each episode consists of 
seamless frames as asserted in the March 23 Office Action, there is no mention or 
discussion of playback between episodes corresponding to user-selected 
annotations being seamless. As discussed above, Gabbe discloses that a user can 
"jump" through icons on a film strip, but there is no mention that annotations 
and/or portions of the media stream corresponding to such annotations are 
provided seamlessly when performing such jumping. 
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Thus, for at least these reasons, Applicant respectfully submits that claim 
73 is allowable over Gabbe 

Regarding amended claim 82, Applicant respectfully submits that, similar 
to claim 59 discussed above, Gabbe does not disclose or suggest saving 
information regarding the media stream segments as an additional new media 
stream as recited in amended claim 82. Applicant respectfully submits that 
nowhere is there any mention or discussion in Gabbe of saving information 
regarding the media stream segments as an additional new media stream as recited 
in amended claim 82. Thus, for at least these reasons, Applicant respectfully 
submits that amended claim 82 is allowable over Gabbe. 

Given that claims 2-6 depend from claim 1, claim 13 depends from claim 
12, claims 29-33 depend from claim 28, claims 35-37 depend from claim 34, 
claims 39-41 depend from claim 38, claims 45-47 depend from claim 44, claims 
49-51 depend from claim 48, claim 53 depends from claim 52, claims 55-58 and 
60-62 depend from claim 54, claims 64-67 and 69-72 depend from claim 63, 
claims 74-81 depend from claim 73, and claims 83-86 and 88-91 depend from 
amended claim 82, Applicant respectfully submits that claims 2-6, 13, 29-33, 35- 
37, 39-41, 45-47, 49-51, 53, 55-58, 60-62, 64-67, 69-72, 74-81, 83-86, and 88-91 
are likewise allowable over Gabbe for at least the reasons discussed above 
regarding their respective independent base claims. 

Applicant respectfully requests that the §102 rejections be withdrawn. 
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Conclusion 

Claims 1-6, 12, 13, 28-86, and 88-91 are in condition for allowance. 
Applicant respectfully requests reconsideration and issuance of the subject 
application. Should any matter in this case remain unresolved, the undersigned 
attorney respectfully requests a telephone conference with the Examiner to resolve 
any such outstanding matter. 



Respectfully Submitted, 





Reg. No. 38,318 
(509) 324-9256 
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